Earth system models (ESMs) are critical for weather forecasting, tracking weather extremes, and supporting impact studies. In particular, numerical weather prediction (NWP) methods track surface and atmospheric data by dissecting the Earth's surface into grids, tracking variables of interest (e.g., temperature, wind speed, direction) as scalar/vector fields, and numerically solving partial differential equations (PDEs) to either physically interpolate into unknown regions or temporally evolve the model--a process known as reanalysis [20, 15]. Historical reanalysis datasets such as ERA5, MERRA-2, and NCEP primarily consist of coarse-scale grid resolutions of 31 31 km to 500 500 km collected by weather stations, aircrafts, and meterological satellites [5, 6, 10]. However, climate simulations at finer resolutions down to 2 2 km are critical for understanding short-term forecasting (nowcasting and medium-range forecasting) and predicting localized weather extremes described by highly resolved fields. As manual collection of such high-resolution data on a global scale is too resource-intensive, global climate models (GCMs) perform downscaling to increase the resolution of surface data by employing two general types of techniques: dynamical and statistical downscaling [3, 11].

In science and engineering, machine learning techniques are increasingly successful in physical systems modeling (predicting future states of physical systems). Effectively integrating PDE loss as a constraint of system transition can improve the model's prediction by overcoming generalization issues due to data scarcity, especially when data acquisition is costly. However, in many real-world scenarios, due to sensor limitations, the data we can obtain is often only partial observation, making the calculation of PDE loss seem to be infeasible, as the PDE loss heavily relies on high-resolution states. We carefully study this problem and propose a novel framework named Re-enable PDE Loss under Partial Observation (RPLPO). The key idea is that although enabling PDE loss to constrain system transition solely is infeasible, we can re-enable PDE loss by reconstructing the learnable high-resolution state and constraining system transition simultaneously. Specifically, RPLPO combines an encoding module for reconstructing learnable high-resolution states with a transition module for predicting future states. The two modules are jointly trained by data and PDE loss. We conduct experiments in various physical systems to demonstrate that RPLPO has significant improvement in generalization, even when observation is sparse, irregular, noisy, and PDE is inaccurate.

In the control problems of the PDE systems, observation is important to make the decision. However, the observation is generally sparse and missing in practice due to the limitation and fault of sensors. The above challenges cause observations with uncertain quantities and modalities. Therefore, how to leverage the uncertain observations as the states in control problems of the PDE systems has become a scientific problem. The dynamics of PDE systems rely on the initial conditions, boundary conditions, and PDE formula. Given the above three elements, PINNs can be used to solve the PDE systems. In this work, we discover that the neural network can also be used to identify and represent the PDE systems using PDE loss and sparse data loss. Inspired by the above discovery, we propose a Physics-Informed Representation (PIR) algorithm for multimodal policies in PDE systems' control. It leverages PDE loss to fit the neural network and data loss calculated on the observations with random quantities and modalities to propagate the information of initial conditions and boundary conditions into the inputs. The inputs can be the learnable parameters or the output of the encoders. Then, under the environments of the PDE systems, such inputs are the representation of the current state. In our experiments, the PIR illustrates the superior consistency with the features of the ground truth compared with baselines, even when there are missing modalities. Furthermore, PIR has been successfully applied in the downstream control tasks where the robot leverages the learned state by PIR faster and more accurately, passing through the complex vortex street from a random starting location to reach a random target.

We introduce a general framework for solving partial differential equations (PDEs) using generative diffusion models. In particular, we focus on the scenarios where we do not have the full knowledge of the scene necessary to apply classical solvers. Most existing forward or inverse PDE approaches perform poorly when the observations on the data or the underlying coefficients are incomplete, which is a common assumption for real-world measurements. In this work, we propose DiffusionPDE that can simultaneously fill in the missing information and solve a PDE by modeling the joint distribution of the solution and coefficient spaces. We show that the learned generative priors lead to a versatile framework for accurately solving a wide range of PDEs under partial observation, significantly outperforming the state-of-the-art methods for both forward and inverse directions. See our project page for results: jhhuangchloe.github.io/Diffusion-PDE/.

The use of machine learning in fluid dynamics is becoming more common to expedite the computation when solving forward and inverse problems of partial differential equations. Yet, a notable challenge with existing convolutional neural network (CNN)-based methods for data fidelity enhancement is their reliance on specific low-fidelity data patterns and distributions during the training phase. In addition, the CNN-based method essentially treats the flow reconstruction task as a computer vision task that prioritizes the element-wise precision which lacks a physical and mathematical explanation. This dependence can dramatically affect the models' effectiveness in real-world scenarios, especially when the low-fidelity input deviates from the training data or contains noise not accounted for during training. The introduction of diffusion models in this context shows promise for improving performance and generalizability. Unlike direct mapping from a specific low-fidelity to a high-fidelity distribution, diffusion models learn to transition from any low-fidelity distribution towards a high-fidelity one. Our proposed model - Physics-informed Residual Diffusion, demonstrates the capability to elevate the quality of data from both standard low-fidelity inputs, to low-fidelity inputs with injected Gaussian noise, and randomly collected samples. By integrating physics-based insights into the objective function, it further refines the accuracy and the fidelity of the inferred high-quality data. Experimental results have shown that our approach can effectively reconstruct high-quality outcomes for two-dimensional turbulent flows from a range of low-fidelity input conditions without requiring retraining.

This is reflected in et al., 2021). Despite the success of PINNs, it is known that several recent studies on characterizing the "failure PINNs sometimes fail to converge to the correct solution modes" of PINNs, although a thorough understanding in problems involving complicated PDEs, as reflected in of the connection between PINN failure several recent studies on characterizing the "failure modes" modes and sampling strategies is missing. In of PINNs (Wang et al., 2021; 2022c; Krishnapriyan et al., this paper, we provide a novel perspective of failure 2021). Many of these failure modes are related to the susceptibility modes of PINNs by hypothesizing that training of PINNs in getting stuck at trivial solutions acting PINNs relies on successful "propagation" of as poor local minima, due to the unique optimization challenges solution from initial and/or boundary condition of PINNs. In particular, training PINNs is different points to interior points. We show that PINNs from conventional deep learning problems as we only have with poor sampling strategies can get stuck at access to the correct solution on the initial and/or boundary trivial solutions if there are propagation failures, points, while for all interior points, we can only compute characterized by highly imbalanced PDE residual PDE residuals. Also, minimizing PDE residuals does not fields. To mitigate propagation failures, we propose guarantee convergence to a correct solution since there are a novel Retain-Resample-Release sampling many trivial solutions of commonly observed PDEs that (R3) algorithm that can incrementally accumulate show 0 residuals. While previous studies have mainly focused collocation points in regions of high PDE on modifying network architectures or balancing loss residuals with little to no computational overhead.

Therefore, it has attracted an increasing amount of attention to combine Physics-informed Neural Networks (PINNs) have machine learning techniques for solving PDEs. Physicsinformed recently achieved remarkable progress in solving Neural Network (PINN) (Raissi et al., 2019) is Partial Differential Equations (PDEs) in various one of the representative approaches that approximate solutions fields by minimizing a weighted sum of PDE loss by training neural networks to minimize a weighted and boundary loss. However, there are several sum of PDE loss and boundary loss -- the former is induced critical challenges in the training of PINNs, including from differential equations while the latter is induced the lack of theoretical frameworks and from boundary and initial conditions. PINN has shown the imbalance between PDE loss and boundary its effectiveness in various sophisticated cases, which has loss. In this paper, we present an analysis of been applied in various fields including fluids mechanics second-order non-homogeneous PDEs, which are (Raissi et al., 2020; Sun et al., 2020), and bio-engineering classified into three categories and applicable to (Sahli Costabal et al., 2020; Kissas et al., 2020).

Physics-informed Neural Networks (PINNs) have recently emerged as a principled way to include prior physical knowledge in form of partial differential equations (PDEs) into neural networks. Although PINNs are generally viewed as mesh-free, current approaches still rely on collocation points within a bounded region, even in settings with spatially sparse signals. Furthermore, if the boundaries are not known, the selection of such a region is difficult and often results in a large proportion of collocation points being selected in areas of low relevance. To resolve this severe drawback of current methods, we present a mesh-free and adaptive approach termed particle-density PINN (pdPINN), which is inspired by the microscopic viewpoint of fluid dynamics. The method is based on the Eulerian formulation and, different from classical mesh-free method, does not require the introduction of Lagrangian updates. We propose to sample directly from the distribution over the particle positions, eliminating the need to introduce boundaries while adaptively focusing on the most relevant regions. This is achieved by interpreting a non-negative physical quantity (such as the density or temperature) as an unnormalized probability distribution from which we sample with dynamic Monte Carlo methods. The proposed method leads to higher sample efficiency and improved performance of PINNs. These advantages are demonstrated on various experiments based on the continuity equations, Fokker-Planck equations, and the heat equation.